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ABSTRACT 

At the urging of the U.S. Department of Education, regional accrediting 
organizations have, during the last few years, placed much greater emphasis on 
implementation of assessment as a requirement for accreditation. Assessment serves two 
important purposes: institutional survival and improvement of student achievement. Of these 
two, survival is the most important. When an institution is preparing for a review by an 
accrediting agency, it may be prudent to minimize or postpone important, but nonessential, 
and sometimes contentious and divisive discussions about discipline-related assessment 
theories and pedagogical best practices. 

This article summarizes and interprets the accreditation experience, and its aftermath, 
within the professional writing program at the University of Houston-Downtown. 
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INTRODUCTION 

The primary audience for this article is faculty and administrators at U.S. colleges and 
universities where accreditation reaffirmation by a regional accreditation agency is upcoming 
and where the commitment to assessment and development of sound assessment practices 
requires additional attention. 

This article 

1. Argues that when survival is at stake, assessment must have a laser-like focus, and that 
potentially important, but nonessential deliberations and initiatives to improve learning 
outcomes, which are susceptible to “analysis paralysis,” should be suspended until 
survival has been secured 

2. Summarizes various approaches to the assessment process 

3. Summarizes the most recent accreditation reaffirmation experience and subsequent 
developments at University of Houston-Downtown (UHD) 

BACKGROUND 

Assessment of higher education is an ongoing, politically charged, and contentious 
issue. The contenders include the United States Department of Education (DOE), Congress, 
accreditors, and colleges (Field p. 1). The DOE, through the National Advisory Committee 
on Institutional Quality and Integrity (NACIQI), can influence the accrediting agencies and 
has drafted them to serve, in part, as proxies for the enforcement of DOE-promulgated 
assessment practices when conducting accreditation reviews of institutions of higher 
education. 

Accreditation agencies are nongovernmental entities, but the DOE can exert 
considerable influence on them because accreditors must meet the DOE criteria for 
measuring the quality of an institution. NACIQI periodically reviews all federally recognized 
accrediting agencies on behalf of the DOE to determine if the accrediting agencies’ standards 
meet the DOE criteria. (Basken p. 1) 

In 1965 the federal government set accreditation by a DOE-recognized accrediting 
agency as a prerequisite for students’ eligibility for federally guaranteed loans. Because these 
loans are an essential source of income for most colleges, accreditation became a requirement 
for the survival of many colleges. (Farrell p. 25) 

The struggle for control over the terms of assessment as a component of accreditation 
has contributed to uncertainty about what accreditors want from colleges (Schmadeka). The 
current emphasis on assessment as an increasingly important component of accreditation can 
blur an important distinction at a critical time. Because assessment is usually undertaken as a 
matter of necessity for institutional survival, but also has the potential to help institutions 
improve education outcomes (Banta p. 3), it is important to recognize that the focus of 
assessment for survival may different from the focus of assessment to improve educational 
outcomes. 

Each discipline has leading scholars who articulately argue the importance, merits, 
and limitations of particular, and sometimes controversial, discipline-related theory and 
pedagogical best practices. Faculty and program administrators are often invested in these 
theories and best practices and vigorously participate in the discussion, development, 
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research, and implementation of such theories and best practices, which can lead to a 
principled and contentious environment when making decisions about the establishment and 
measurement of learning outcomes. Because, as J. Allen (2004) notes there is a tendency to 
formulate assessment strategies that try to do too much (p. 98), as the time for reaffirmation 
of accreditation approaches, important, but nonessential deliberations and initiatives to 
improve outcomes, which are susceptible to “analysis paralysis,” may have to be temporarily 
suspended until survival has been secured. Some may argue that advocacy for such a 
suspension is cynical or compromises intellectual integrity. Those who see this dilemma as 
“institutional Maslowianism” (Hierarchy of Needs) might argue that when a gun is being held 
to one’s head, it is prudent to surrender one’s valuables. To thrive, one needs to first survive 
(Maslow). 

ASSESSMENT PROCESS 

Various authorities (M. Allen, Angelo, Huba and Freed, Palomba and Banta, 
Williams) describe educational assessment is an iterative, data-driven process that is intended 
to improve learning. The current assessment jargon describes the completion of one iteration 
as “closing the loop.” If the loop is not closed, the assessment is incomplete. 

Authorities differ in their descriptions of the steps required to close the loop. The 
differences are not about the loop, but rather about the approach and the size of the steps to 
traverse the loop. 

For example, Williams (2008) describes assessment as a four-step process: 

1. Specify the learning outcomes (LOs). LOs are what students should know and 
be able to do as a result of completing a unit of learning (lesson, course, degree 
program). 

2. Specify how to measure the extent to which students have achieved the LOs. 
The measures must be carefully selected or designed so that they actually measure 
the LOs, rather than something else. The LOs and their corresponding means of 
measurement must be closely aligned. 

A good rule of thumb is to specify at least one direct measure and one indirect 
measure for each LO. Direct methods of assessment require students to produce 
work so reviewers can assess how well students meet expectations. Indirect 
methods of assessment allow students to report their perceptions of their learning 
experiences. 

3. Collect data on the extent to which students have achieved the LOs. 

4. Study the assessment results and, as needed, make changes to improve student 
learning (Circle of Assesssment section). 

Banta (2004) advocates a tailored approach to developing an effective assessment 
program. She emphasizes the necessity of adapting the development to the unique context of 
the institution. 

Unfortunately there is no silver bullet, no magic potion, no single step-by- 
step approach that will provide easy answers to the fundamental question about 
how to get started in outcomes assessment. This is because every setting is 
different and requires its own unique approach, taking into account the mission 
and goals of the campus and of the individual unit undertaking assessment, the 
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expertise and interests of those to be involved in achieving the mission and goals 
and in guiding assessment, the prior history of evaluation initiatives in the unit, 
and the resources available to implement assessment, to name just a few of the 
relevant variables (p. 1). 

Rather than identifying steps in the process of closing the loop, Banta identifies three 
phases: Planning, Implementing, and Improving and sustaining . She suggests that writing a 
multi-year plan is often, but not necessarily, the appropriate place to start. Banta explains that 
planning for effective assessment requires stakeholder involvement, includes a written plan, 
is based on explicit program objectives, begins when the need is recognized, and allows 
enough time for development. She emphasizes, “Time is crucial” (p. 2). 

Advocates of Banta’s approach might consider it to be more finessed, sophisticated, 
and efficient, while advocates who favor the more prescriptive regimens, such as those of 
Williams, might view Banta’s approach, because of its fluidity, as less appropriate for those 
without a sophisticated understanding of both assessment and the unit being assessed, which 
in this case is a technical communication program. 


TECHNICAL COMMUNICATION PROGRAM ASSESSMENT 

The increased emphasis on institutional and program assessment by the regional 
accrediting bodies, and others, makes program assessment, including of technical 
communication programs, increasingly important. 

The importance of assessment to technical communication programs notwithstanding, 
more guidance and understanding of assessment of technical communication at the program 
level is needed. Given the relatively recent shift in emphasis on program level assessment and 
the relatively long lead time required to develop, implement, and evaluate the effectiveness of 
assessment efforts, the current lack of guidance and understanding of best practices for 
assessment of technical communication at the program level is understandable. 

It is encouraging to note that technical communication educators are addressing these 
deficiencies. For example, J. Allen has provided examples of how matrices and rubrics can 
be used to realize widely accepted characteristics of effectiveness in the assessment of 
technical communication programs (pp. 103-06). Salvo and Ren (2007) have offered a model 
for participatory assessment that is based on research in participatory, user-centered design. 
Carengie (2007) proposes development of a contextual model for program assessment. As a 
result of such efforts, guidance, best practices, and various permutations of assessment 
instruments are emerging. 

Inherent in this state of emergence is a degree of ambiguity about the appropriate 
constituents and means for the effective assessment of a technical communication program. 
This state of emergence and ambiguity is, in part, a consequence of the state of writing 
assessment, and, in part, a result the increased emphasis on program assessment by the 
regional accrediting bodies, and others. 

Theories, principles, and best practices of writing assessment have changed during the 
last 60 years, and will continue to change. Scholars and researchers advance and argue 
divergent approaches to important aspects of writing assessment. However important these 
deliberations may be, in the context of an impending review of program assessment by a 
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regional accrediting body, they are less important than providing sufficient evidence to assure 
the accreditor than an assessment plan for the program exists and is being executed. 


UHD 


The focus of the UHD accreditation reaffirmation by the Southern Association of 
Colleges and Schools (SACS) was on documentary evidence of 

• a viable, multi-step plan to close the loop 

• implementation of the plan 

UHD has a long history of conducting assessment, but assessment was not consistent 
across colleges, departments, or programs. Of great importance to SACS was the fact that the 
loop was not always closed; the assessment cycle of data collection, evaluation, and 
modification to improve learning outcomes was not always completed. SACS conducted an 
on-site review of UHD from March 7-9, 2006, and, in a letter dated January 9, 2007, made 
several recommendations, including the following: “The institution should provide evidence 
that student learning outcomes are being assessed for each of the educational programs and 
that the assessment data are used to enhance the quality of the programs” (B. Wheelan, 
personal communication). 

Mobilization 

After further reporting and correspondence resulted in a stalemate, the UHD upper- 
level academic administrators, with the help of an assessment consultant finalized a plan of 
action during the first weeks of 2009. The faculty senate met to both chastise these upper- 
level administrators for their role in the failure to avoid the impass and to express the 
willingness and availability of the faculty to do whatever was necessary to help bring the 
situation to a satisfactory conclusion. 

The jeopardy to accreditation contributed to a crisis mentality, which turned out to be 
a mixed blessing. On one hand, it provided the significant advantages of a focused approach 
and highly motivated participants; on the other hand, it limited the breadth of stakeholder 
participation and depth of widespread deliberation among faculty about the range and 
desirability of various assessment options. 

The immediate strategy of the UHD assessment effort was clear: satisfy SACS. 

J. Allen (2004) recommends a streamlined approach to assessment strategy development (p. 
98). The approach at UHD was nothing, if not streamlined. So much so that the education 
and thoughtful deliberation of stakeholders was necessarily truncated. It was not the fear of 
the anarchy of participation (Latour 1999), but rather the fear of failure, that limited the 
implementation of participatory assessment, as described by Salvo and Ren (2007). There 
was little time for democratic deliberations or academic arguments. In their place was a 
succinct strategy and a drop dead date. 

Banta (2004) editorializes, . . stakeholders are much more likely to become engaged 
if they can perceive assessment as an integral part of a process they value such as curriculum 
development, peer review, or personal scholarship” (p. 3). At UHD, stakeholders became 
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engaged in assessment because they feared the consequences of failing to satisfy SACS. 
Motivation among administrators, faculty, and staff was high. 

The deans led the efforts within their colleges to develop the documentation required 
for SACS. Each department recruited one or more individuals to lead the effort at the 
program level. The deans tasked the department-level liaisons with the development of LOs 
and rubrics, and the identification of indirect measures with which to assess the achievement 
of the LOs. Rubrics were a good choice for measuring the diverse LOs to be identified by the 
departments. As M. Allen (2004) states, “Rubrics can be used to classify any product or 
behavior, such as essays, research reports, portfolios, works of art, recitals, oral presentations, 
performances, and group activities” (p. 138). 

Results 

The jeopardy to the University’s accreditation was unsettling, the topic of much 
speculation, and some finger pointing. The effort to clarify, implement, and document the 
level of assessment required to satisfy SACS was urgent, intense, and far-reaching. SACS 
reaffirmed the accreditation of UHD at the 11 th hour. 

The effort to institutionalize meaningful assessment continues. The institution’s 
mission statement has been rewritten. A director of assessment has been hired. Explicit LOs 
for programs across the University now exist, as do various means for measuring the 
achievement of the LOs. It seems that faculty and administration now have a somewhat 
greater understanding of the nature and growing importance of assessment, which may result 
in an increased receptivity to participate in various assessment activities, as will the proposal 
of the English Department Rank and Tenure Committee to add the tenure requirement of “. . . 
assessing success in meeting learning outcomes and adjusting teaching practice as assessment 
indicates is advisable” (p. 1). It is too soon to tell if assessment will become part of the 
institutional culture at UHD or to identify the impact of assessment on the achievement of 
learning outcomes, but at this time, it appears that significant momentum is building for the 
creation of a culture of assessment at UHD. 

The urgency of the situation intruded on the process at UHD and may have provided 
some justification for ignoring principles of good practice for assessing higher education, as 
identified by Astin, et al., and published through the American Association of Higher 
Education. At UHD, it seemed that 

• Conceptualization was a group effort, albeit a very small group, which expedited the 
conceptualization process. 

• Participation by external stakeholders, such as actual and potential benefactors, was 
restricted, possibly to protect both ongoing development efforts and the University’s 
reputation. 

• Long-term sustainability of the assessment effort was uncertain. There was reason to 
wonder what would happen with assessment if the immediate threat could be averted. 
The net effect of the urgency was to accelerate the development of a successful 

response to the immediate threat, possibly at the expense of achieving what is ostensibly the 
longer term objective: creating a sustainable culture of effective assessment. 
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Role of the Professional Writing Program 

Long before the most recent accreditation visit by SACS, the professional writing 
(PW) program developed explicit LOs, specified measures, consistently collected data, and, 
with somewhat less consistency, analyzed the data to identify program strengths and 
weaknesses, and made program modifications in response to the results of the analysis. 

At the time of the accreditation review of UHD by SACS, the evidence suggests that 
SACS was more concerned that UHD was closing the loop and less concerned about 
precisely how it was closing the loop, as long as data was being collected through at least one 
direct and one indirect means. At the PW program level, this suggests that, whichever side of 
a writing assessment issue a program’s administration or faculty may support was of little 
consequence to SACS. From the narrow perspective where the success of writing assessment 
is defined only in terms of receiving accreditation renewal, it was not necessary to justify the 
methodology, thereby keeping the primary focus on accreditation, rather than the important, 
but untimely and sometimes contentious questions and issues surrounding writing 
assessment. 

The direct measure was a rubric that was applied annually to the portfolio required of 
each PW major for graduation. The PW faculty believed, as argued by Elbow and Belanoff 
(1986), that portfolio evaluation did not suffer the inherent limitations of proficiency exams, 
which curtail reading, reflection, and discussion, and do not allow for the more balanced 
picture of a student’s proficiency afforded by the multiple writing samples of a portfolio. 

Prior to the annual collaborative portfolio evaluation of 2009, the PW faculty met for 
an “interrater reliability” session. The session was interesting for two reasons. First was the 
disparate evaluations of student-written recommendation reports, and second was the 
response of the group to the disparities, which was to ask the individual evaluators whose 
scores were furthest from each other to explain their scoring. These explanations revealed 
meaningful differences between faculty in their frames of reference (raters’ contexts), which 
resulted in various interpretations of the rubric’s evaluation criteria, thus the disparate scores. 
However, there was little pressure to achieve numerical consistency. The focus seemed to be 
on understanding differences, rather than reconciling them. The net result seemed to support 
Broad’s (1994) contention that “ . . . getting people to discuss their evaluations prior to 
scoring helps to open up the conversation and make room for divergent perspectives that are 
often squeezed out under the pressure for numerical agreement” (p. 291). 

Data was collected during the annual collaborative portfolio review. Using the rubric, 
three PW faculty members evaluated each portfolio. Unfortunately, deliberations by the PW 
faculty about the implications of the rubric data for program modification were secondary to 
the portfolio evaluation, which was an end unto itself, and somewhat overshadowed the use 
of the data to identify and implement program modifications. Consideration of questions 
about the limitations and best uses of portfolios in the effective evaluation of writing, such as 
those raised by Huot (1994), were on the periphery of the portfolio review session, as were 
concerns about validity and reliability (Huot, Yancey, White, and others) of the assessment 
process. This was not due to lack of concern about the larger issues, but rather the immediacy 
of the need to complete the task at hand. If there is a time for reflection and a time for action, 
that was definitely a time for action. The undergraduate PW program coordinator not only 
documented the ongoing use of the rubric to evaluate portfolios, but also summarized the 
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conclusions and resulting recommendations of the faculty abbreviated deliberations about the 
implications of the rubric data for program changes needed to remedy deficiencies identified 
through the portfolio evaluation. 

The indirect measure was mandatory student evaluations of courses and faculty 
performance. Institutional policy required that the student evaluations be incorporated into 
the annual performance evaluations of faculty. Institutional policy did not require that the 
implications of the student evaluations for program modifications receive consideration. 

As J. Allen (2004) said is commonplace, assessment of the professional writing (PW) 
program was important not only in its own right, but also for its role in providing a writing 
intensive course that helps fill the general education requirement for writing proficiency (95). 
Because of the assessment deficiencies SACS identified in general education, the PW 
program curriculum committee was also tasked with providing assessment documentation to 
support the general education program. PW offers a writing-intensive course, ENG 3302 
Business and Technical Report Writing, which helps satisfy the general education 
requirement for writing proficiency. The major ENG 3302 project is a recommendation 
report. The curriculum committee developed a rubric for use with recommendation report. 
During the development of the rubric for ENG 3302, the curriculum committee graded a set 
of recommendation reports without using the rubric, and then graded the same set of reports 
using the rubric. Grades were more uniform with the use of the rubric than without, which 
prompted some discussion about the advantages and limitations of the rubric. The concern 
was that the rubric did not accommodate committee members’ differing perceptions of the 
relative importance of the various LOs. The resolution was to encourage faculty to use the 
rubric without modifications, while allowing modifications by individual faculty members as 
they felt compelled. The committee adopted the strategy of embedding the rubric in the 
scoring of the assignment, as suggested by Walvoord and Anderson (1998), and then annually 
compiling and reviewing the data across ENG 3302 courses. The looming SACS deadline 
prevented the committee from considering the use of other types of measures, for example, 
focus groups, as Eubanks and Abbott suggested (2003), which is inconsistent with the advice 
of Hamilton and Banta (2008), who warn, “To retain control of assessment at the institutional 
level, faculty and administrators are going to have to learn as much as possible about multiple 
modes of assessment and what each mode can tell us” (p. 28). 

Aftermath 

The effort to institutionalize meaningful assessment continues. The institution’s 
mission statement has been rewritten. A director of assessment has been hired. Explicit LOs 
for programs across the University now exist, as do various means for measuring the 
achievement of the LOs. It seems that faculty and administration now have a somewhat 
greater understanding of the nature and growing importance of assessment, which may result 
in an increased receptivity to participate in various assessment activities, as will the proposal 
of the English Department Rank and Tenure Committee to add the tenure requirement of . . 
assessing success in meeting learning outcomes and adjusting teaching practice as assessment 
indicates is advisable” (p. 1). It is too soon to tell if assessment will become part of the 
institutional culture at UHD or to identify the impact of assessment on the achievement of 
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learning outcomes, but at this time, it appears that significant momentum is building for the 
creation of a culture of assessment at UHD. 

It is encouraging to note that significant initiatives have been launched at UHD to 
create a sustainable culture of assessment. These include training for interested stakeholders 
and explicit directives to develop and implement additional sustainable means of program 
evaluation, based on program LOs. For example, the 2010 annual evaluations by PW faculty 
of student portfolios required faculty to complete an additional rubric, which is intended to 
assess how well the PW program achieved two of its eight LOs (“Engage in projects for real 
clients” and “Conduct, evaluate, and report research and provide accurate citations”). The 
rubric data will be compiled, analyzed, and used to help identify any program deficits and the 
changes required to address those deficits. The program’s performance against all eight LOs 
will be assessed during a five-year cycle, with different LOs being assessed each of the five 
years. 

CONCLUSION 

DOE pressure has compelled the regional accrediting bodies to put greater emphasis 
on assessment. These bodies want more than a plan for institutional and program assessment. 
They require sufficient evidence of ongoing and effective institutional and program 
assessment. Institutions and programs that fail to provide such evidence jeopardize the 
reaffirmation of their accreditation. 

It takes time to develop and document the institutionalization of effective assessment 
practices. To protect institutional and program reputations and financial viability, faculty and 
administrators of institutions and programs that are subject to an accreditation review by a 
regional accrediting body would do well to take a step back from dissention about the details 
of assessment and focus on the completing the steps required to satisfy the accrediting body. 
Once the accreditor is satisfied, the pursuit of improved learning outcomes can resume with 
full vigor. To thrive, one must first survive. 
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